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Abstract 

We show that the entropy of the distribution of avalanche lifetimes in the Kinouchi-Copelli model always 
achieves a maximum jointly with the dynamic range. This is noteworthy and nontrivial because while 
the dynamic range is an equilibrium average measure of the sensibility of a sensory system to a stimulus, 
the entropy of relaxation times is a purely dynamical quantity, independent of the stimulus rate, that can be 
interpreted as the efficiency of the network seen as a communication channel. The newly found optimization 
occurs for all topologies we tested, even when the distribution of avalanche lifetimes itself is not a power 
law and when the entropy of the size distribution of avalanches is not concomitantly maximized, strongly 
suggesting that dynamical rules allowing a proper temporal matching of the states of the interacting neurons 
is the key for achieving good performance in information processing, rather than increasing the number of 
available units. 

PACS numbers: 64.60.av, 89.70.Cf, 87.85.dq 
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In the end of the last century appeared the first claims on the criticality hypothesis, stating 
that some biological systems could evolve towards the edge of chaos [[j]-§p, the critical surface 
separating two phases in an abstract space of parameters. The heuristic justification for this hy- 
pothesis is that adaptation of biological systems would be guided by selective pressures favoring 
the optimization of some key attributes, e.g. the capacity of sensing environments. No matter 
how appealing was the proposal, earlier work [[I]-§p relied only on simulations and the lack of 
experimental validation combined with some criticisms [|7|] turned down the theory for some time. 
However, captivating researchers on brain dynamics, the idea acquired a wholly new motivation 
[§]. Indeed not only new theoretical evidence of increased computational performance at criti- 
cality showed up recently []9|-[T2|] but mainly the observation of power-law behavior of neuronal 
avalanches in cortical networks both in vitro [ ]T3| , [14| ] and in vivo [p~5] — P~7|] constituted stronger- 



than-ever evidence of the relevance of the edge of chaos to the operation of biological systems. 
There was some debate regarding the proper characterization of the recorded power laws [[[8|, [19|] 
and whether criticality would be the sole explanation for the observed scale invariance [ |20| ] but 
this time the criticality hypothesis is standing up criticisms [§, ^p. Remarkably, some experiments 
have explicitly revealed maximized quantities as information transmission [ fT5| , ETJ ], information 



capacity pT|], synchronizability [ [22"[ ] and dynamic range [ [231 ] m cortical networks at a critical 
condition. 

The dynamic range is a sensibility measure associated to the high-slope region of a tuning 
curve (also response function, the stimuli-response relationship characterizing a sensory network), 
where nearby stimuli can be most easily discriminated since small changes in stimulus lead to high 
changes in the firing response. Theoretical work [[TTJ] indicated that the dynamic range should be 
maximized in sensory systems when the topology of the network was set in a specific condition, 
critical for the signal propagation among interacting excitable neurons. A beautiful marriage of 
theoretical prediction with experimental confirmation happened when an optimal dynamic range 
was found [ ]23| ] in cortex slice cultures with a proper balance between excitatory and inhibitory 
interactions achieved through pharmacological manipulation. 



In this study, we employ simulations to show that, when the Kinouchi-Copelli (KC) model [ |TT| ] 
is tuned at the edge of chaos, the Shannon entropy of its avalanche lifetime statistics (hereafter 
information efficiency) is always jointly maximized with the dynamic range, both in the original 
[ [TT] ] random graph topology and alternatives. We note that the information capacity (the entropy 
of avalanche size distribution) does not always exhibit such a critical optimization. Indeed, we 



believe that information efficiency rules the behavior of the dynamic range and outweighs by far 
the relevance of information capacity in determining the information processing properties of a 
sensory network. Two previous works discussed critical optimization of entropies of avalanches 
sizes. The first one [E3p was a theoretical study of measures of information propagation in Boolean 



networks and the second one was the discovery of optimal information capacity in critical cortical 
networks [|ZT|]. None of them have raised the possibility that the information efficiency could be 
maximized instead of capacity. 

The model. — We start giving a concise description of the KC model JTT| ] : each of the N neurons 
is a cellular automaton that can be at states (quiescent/excitable), 1 (excited) or 2,--- ,m— 1 
(refractory states). The neurons are arranged as a weighted undirected graph with mean degree K. 
The sequential transitions 1 — >• 2, 2 — > 3, m — 2 — > m — 1 and m — 1 — > are deterministic. On 
the other hand, the transition — > 1 happens for neuron k if either (i) a spark of any of its excited 
neighbors j reaches it, with probability p^j drawn from an uniform distribution in [0,p max ]> or (ii) 
if k gets an external stimulus, modeled by a Poisson process with rate r, resulting in an excitation 
probability X = 1 — exp (rAt) at each time interval At. All other transitions are forbidden. 

By setting p max = 2o/K, the mean number of excitations an excited neuron could generate 
in one time step if all its neighbors were quiescent is a, namely, the average branching ratio. 
Given the fraction of excited nodes p t at time t, the proper psychophysical response of the system 
is the average activity F = T~ l Yj=iPt- Consequently, the response is a function F = F(r) of 
the stimulus rate r. The dynamic range A is defined in decibels as A = 101og(ro.9/ro.i), where 
F(r x ) = F min +x[F maK -F min ],x G [0, 1], F max = F(oo) is the satured response and F^ = F(r -»■ 0) 
is the spontaneous activity. Kinouchi and Copelli showed that self-sustained activity is possible if 
a is greater than o c = 1 [|TT|], so that F m i n plays the role of the order parameter in a phase transition 
in the neural activity with o as a control parameter. They have also found a critical optimization 
for A. 

In [ |TT1 ] the authors studied only the Erdos-Renyi topology (ERT) with a fixed number NK/2 of 
connections and focused on characterizing the maximization of the dynamic range, as did further 



works on alternative topologies [|25|-|27|]. They did not dwell on exploring the bursts of activity 
(avalanches) generated by their model, although stating that critical networks exhibit both large 
variance of avalanche lifetimes and a power-law distribution for avalanche sizes with the classical 
exponent —3/2 



In this work, we study the avalanches exhibited by the KC model implemented on both the ERT 



and the Barabasi-Albert [ pO| | topology (BAT). Unless explicitly stated otherwise, the simulations 
were performed with N =10 5 and K = 10. Given a randomly generated representant of a topology, 
with chosen average connectivity K and average branching ratio a, we randomly choose a neuron 
of the network to be initially excited while all others are quiescent and record both the number s 
of neurons that get excited due to that single spark and the number t of consecutive generations 
the network remained active. We repeat this procedure a large number of times in order to get 
the distributions {p s } and {p t } of the size and the lifetime of an avalanche, respectively. In this 
setting, there is no role at all for a stimulus rate. 

Size and lifetime distributions. — Fig. [j] (a) illustrates the critical (at o c = 1) emergence of 
power-law scaling in the bursts of activity, p s ~ s~ 15 and p t ~ t~ 19 , for the ERT (exponents 
estimated with standard techniques [|TJ, [32|]). Since the ERT allows the propagation of almost 
independent branches of activity, this behavior is perfectly compatible with the predictions from 
the theory of branching processes []2~8|, [29|]. Indeed, we will describe elsewhere how that formalism 
predicts the solid lines in the bottom of Fig. [j] (a). 

In Fig. H] (b) we exhibit the avalanche distributions in a BAT. It is harder to estimate these 
distributions and there is much uncertainty in the size distribution, but the bottom of Fig. [1] (b) 
shows clearly the presence of a bump in the lifetime distribution for a = 0.4 (p s ~ s~ 2 and p t ~ 
/~ 2 ' 9 in the first decades) and strongly suggests power-law behavior for slightly smaller values 
of a. A naive analysis based on the criterium of a power law as a signature of critical behavior 
would favor the latter against the former, but we remark that the observation of power laws in 
the KC model in this topology demands subsampling to a 2-10% level (not shown, but see also 



Qlq , p3Q) and below we will present results supporting the "bumpy" curve as the critical one. We 
start discussing the relation among the dynamic range and the entropies of the distributions just 
described. 

Information efficiency. — The Shannon entropy H of a distribution {p n }, 

H ({Pn}) = -Y,P" l °SPn, 

n 

is a standard measure of the uncertainty of a stochastic observable and so it is quite obvious it 
should be applied to the analysis of avalanches. We will set n as either s or t to indicate which 
distribution we are talking about. In Fig. [2|, we jointly illustrate the behavior of the dynamic range 
A and of the entropies as functions of o, given K and a topology. 

It was not surprising to observe in Fig. || (a) H({p s }) getting maximized jointly with A in 
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Figure 1. Distributions of (top) burst size and (bottom) activity lifetimes for (a) Erdos-Renyi and (b) 
Barabasi- Albert topologies. In (a), power laws emerge for o c = 1 and the solid lines in the bottom re- 
sult from the theory of branching process (see text). In (b), especially in the bottom, huge avalanches 
become more frequent when a = 0.4. We have evidence that this is the critical condition, despite the lack 
of a power law (see text). 

the ERT, since two previous studies P^l h i n different contexts, reported such an effect for the 
information capacity. Moreover, it seems natural to associate the dynamic range to that entropies 
(as measures of the flexibility of the system in dealing with signals), so that heuristically H({p t }) 
should exhibit critical optimization following A. 

Thus, since the critical optimization of A in BAT's has already been stablished [T25J, the 



lack of a corresponding peak for H({p s }) in Fig. [|(b) came as a complete surprise. Later we will 
discuss how that happens while H({p t }) and A keep getting optimized jointly, this time at a = 0.4. 

Is this optimization really critical! As did the authors of [f25|], we claim so invoking as first 
evidence the plot of the order parameter F m i n of the KC model against the control parameter a in 
the inset of Fig. 0(b). However, our results suggest o c = 0.4, while o c ~ 0.5 was estimated in []25]]. 
Despite the much smaller networks simulated in that work, such a divergence demands an even 
more stringent analysis regarding the critical nature of the position of the peaks in this topology, 
namely, a tentative of data collapse into a scaling function. We emphasize we are not going to take 
the abscissa of the peak as the location of a critical point a priori, since there is no support for 



such a procedure. Quite the opposite, in [[241], f° r instance, stochasticity makes H({p s }) exhibit a 
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Figure 2. Dynamic range (A) and entropies of avalanche size and lifetime distributions as functions of 
the average branching ratio a in (a) Erdos-Renyi and (b) Barabasi- Albert topologies. Concomitant critical 
optimization emerges clearly in (a), but only the lifetime entropy gets optimized jointly with A in (b). The 
green dashed lines are guides for the eyes. Inset: F m ; n vs. a, clearly indicating the critical value a c ss 0.4. 



peak away from criticality. 

We also remark that (i) the deviation of o c from 1 in alternative topologies has been recently 



explained in terms of a spectral analysis [[34], |35|] and nowadays is not surprising at all, (ii) there 
is no critical optimization for BAT's grown node by node (i.e., the information efficiency and 
the dynamic range keep behaving jointly even in such a "charmless" scenery) and (iii) whatever 
be the critical point, it is very clear from Fig. [| (b), mainly from the plot of the information 
efficiency H({p t }), that the joint optimization of H({p t }) and A does not happen at the values of 
o resembling power-law behavior in Fig. [j] (b). 

Critical optimization. — Despite the inset in Fig. [Z] (b) being a classical signature of a phase 
transition, the aforementioned displacement of the critical point prompted us to perform a scaling 
analysis in order to confidently assess the issue of criticality. Let p^(?) be the complementary 
cumulative distribution function (CDF) of the lifetime of an avalanche when there are N neurons 
in the KC model (i.e., Pat(/) is the probability of the duration of an avalanche surviving for at 
least t given a network with N units). Given a topology and a value of o thought to be a critical 
point, we have looked for exponents y and D able to make all the transformed distributions x^Pn(x) 
collapse into a single universal curve when plotted against x/N D , regardless the value of N. Fig. 
|3] illustrates the success of such endeavour, considering the distributions obtained with o c = 1 in 




Figure 3. Instances of (top) CDF's for several system sizes and (bottom) their collapse into a scaling function 
for both (a) Erdos-Renyi and (b) Barabasi- Albert topologies. To collapse the CDF's at the proposed critical 
conditions, we have fit Jer = 0.9, Der = 0.5, Jba = 1-25 and Dba = 1 (see text). The distributions in (b) 



closely resemble the ones presented recently by Dehghani et al [36]. 



the ERT and c c = 0.4 in the BAT. 

There are two more observations supporting our claim of criticality in the absence of pure power 
laws in the avalanches (data not shown). Both preliminary results on the collapse of avalanche 
shapes, as advocated by Q37j ] and implemented in []38|], and an analysis of the extinction probability 
(a branching-process-like study to be discussed elsewhere) point to o c = 0.4 in the BAT. 

We also report complementary observations regarding data not shown in this paper. In contrast 
with other contexts ^5p, our results are qualitatively robust to changes in the distribution of 
edge weights (even for constant weigths). Most alternative topologies [|4(], |T]] leading to more 
realistic degree distributions [^] are qualitatively equivalent to the BAT and exhibit even more 
pronounced bumps. In the uncorrelated version of the configuration model [jflp, bumps are 
still present in the avalanche distributions but both information capacity and efficiency exhibit 
critical optimization. This behavior fits in the phenomenology we describe here, but also suggests 
that the existence of degree correlations in the BAT may be the reason why H({p s }) does not 
maximize in that topology. 

Discussion. — Neuroscientists regularly employ the mutual information as a measure of the sta- 
tistical dependence between stimulus and response, taking into account both specificity (change in 
stimulus implying change in response) and fidelity (low variability given a stimulus) of responses. 
Notwithstanding the relevance of such studies, indispensable for the comprehension of neural cod- 
ing, we remark our aim in this paper is the study of intrinsic, stimulus-free, behavior of sensory 
systems rather than of patterns of response variability. 
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The KC model [ |TT| ] revealed that the stimulus-dependant measure A is optimized precisely at 
o c , a critical point of the self-sustained activity F m [ n , that does not depend at all on the nature of the 
stimulus. If not a theoretical artifact, that phenomenology suggests that generally the behavior of 
actual excitable networks (e.g., sensory systems) could be strongly determined by its stimulus-free 
properties. Accordingly, an optimal dynamic range was observed experimentally at a condition de- 
termined in the absence of stimuli [ ]23| ] and there is further robust evidence that stimulus-evoked 
activity is strongly dependent on the spontaneous (r = 0) firing patterns in the cortex (see refer- 
ences in [p3|]). Therefore, despite it may seem quite plausible per se that the sensibility measured 
by the dynamic range gets optimal when signals do not either fade out fastly (subcritical) or fre- 
netically superimpose themselves (supercritical), there is enough motivation for scrutiny on the 
microscopic mechanisms leading to improvements in information processing capabilities even in 
stimulus-free conditions. 

We firmly believe to have discovered one such mechanism. It seems fairly intuitive to expect 
that a sensory system be more flexible if it disposes of greater variability of the duration of the 
bursts of neural activity it must process. Indeed the support of the avalanche lifetime distribution 
(the interval of lifetimes with positive probabilities) achieves a maximum jointly with the dynamic 
range, while noncritical distributions hold for no longer than two decades (see Fig. p. We have 
chosen the information efficiency as the proper measure of that effect because it takes into account 



not only the magnitude of the reportoire of lifetimes but also a balanced utilization [|3|] of that 
resource for information transmission (from the olfactory bulb to the cortex, for instance). 

It is not evident why such arguments are always valid for the information efficiency but not 
for the capacity. We speculate that an explanation must rely on the relationship between the mi- 
crostructure of the network (motifs) and the dynamics of the excitable units. Progress beyond the 



initial studies on the synchronizability properties of the KC model [E2L 121, FRJ| will be probably 



achieved by deciphering such relationship. As a rule of thumb, greater values of the clustering co- 
efficient plt l should lead to stronger deviations from the —3/2 law. It is also worth mentioning that 
it has already been suggested p7| ] that "the lifetime distributions of neuronal avalanches may carry 
rich information about the local cortical circuit structure" and may exhibit consistent deviations 
from power-law scaling, while the size distribution would be much more well-behaved. Anyway, 
despite our praise of the intrinsic activity, it is definitively worth studying stimuli-dependant fea- 
tures of excitable networks by information-theoretic tools like mutual information and transfer 
entropy. 
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The notion of criticality without power laws may have wide implications in the interpretation 
of observations of neuronal avalanches. Recent experiments exhibiting critical optimization []2T]— 



23Q have described the phase transitions in terms of a control parameter K resembling o but based 
on the tacit assumption that neuronal avalanches are pure power laws. Further investigations are 
necessary to reveal eventual consequences of the breakdown of that hypothesis. Likewise, Ref. 
[ |3"o] ] employed robust statistical techniques to analyze neuronal avalanches in vivo and stand up 
against critical dynamics. However, the CDF's they present are very similar to Fig. ^| (b), so that 
probably their data is ruling out power-law scaling, but not criticality. Finally, distributions pretty 
much like the ones in Fig. [j] (b) have been recently observed in high-resolution experiments in vitro 



[ p8| ] and the bumps were no obstacle for a remarkable data collapse constituting very compelling 



evidence of critical behavior in brain dynamics. 

Summarizing, we studied the avalanches in Kinouchi-Copelli model in a first attempt to figure 
out detailed mechanisms of information transmission in cortical networks. We discovered that, 
in a critical point, the entropy of avalanche lifetime statistics (information efficiency) is always 
maximized jointly with the dynamic range, an important measure of information transmission 
extracted from the psychophysical tuning curves. Our findings fit in the discussions regarding the 
role of criticality in information processing [^J, ^] and the relationship of long bursts of activity 
with the dynamic range [[231], specially because they suggest critical behavior without pure scale 
invariance. 
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